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(57) Abstract 



A method is disclosed for supplying arbitrary types of information in a digital signal conformant to the structures defined in ISO 
standard ISO/IEC 13818-1, known generally as MPEG2 Systems. The method extends Program Stream and Transport Stream Structures 
by application of principles developed for standardization of bidirectional data communications protocols to define a universal messaging 
syntax applicable to all types of information, including Internet data streams, VBI data streams, other types of data streams, as well as 
MPEG2 video, audio, and control; to define a multiplexing structure that is independent of the details of the MPEG2 signal structure; and 
to define a scheme for addressing such enhanced MPEG2 signals, or portions of signals, to individual decoders or to groups of decoders. 
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METHOD FOR UNIVER SAL MESSAGING AND MULTIPLEXING 

Background of the Invention 

The invention relates generally to the transmission of 
digital signals , and more particularly to the transmission 
5 of digital signals in systems containing digital video and 
audio signals conforming to packet-based multiplexed 
communication standards. 

The International Organization for Standardization 
(ISO) recently adopted a standard (ISO/IEC 13818-1) that 

10 addresses the combining of one or more "elementary streams" 
of video and audio, as well as other data, into single or 
multiple streams suitable for storage or transmission. The 
ISO/IEC 13818-1 standard, hereinafter referred to as the 
M MP EG 2 Systems" standard, is described in detail in the ISO 

15 draft document "Generic Coding of Moving Pictures and 
Associated Audio", ISO/IEC JTC1/SC29/WG11 N0801 (13 
November 1994), which is incorporated herein in its 
entirety by reference. The MPEG 2 Systems standard 
specifies syntactical and semantic rules for system level 

20 coding of the elementary streams using packet-oriented 
multiplexes. The MPEG2 Systems standard defines an 
individual coded video, audio or other coded bitstream as 
an "elementary stream". The contents of an elementary 
stream may be broken into a sequence of discrete units, in 

25 which case the elementary stream is structured as a 

Packetized Elementary Stream (PES) . The individual units, 
or packets, are known as PES packets, which can be of large 
and variable size. The MP EG 2 Systems standard defines 
generic structures for PES packet formats and specifies 

30 particular rules for creating PESs from digital video and 
audio elementary streams. Allowance is made for creating 
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i.e., data formats not 



10 



PESs for private data formats, 
defined by the MP EG 2 Systems standard. 

The MP EG 2 Systems standard defines two methods of 
creating a multiplex of PESs. In a Program Stream (PS), 
all components in the multiplex are assumed to belong to a 
single "Program", that is, a collection of elementary 
streams which may sensibly be presented as a unity to a 
user, all components being referenced to a common time 
base, together with certain coordinating control 
information. PES packets from component PESs are 
multiplexed by PES packet. The PS is rigidly structured so 
that at most one PES of a specified format may carry a 
private data elementary stream. 

in a Transport Stream (TS) , the components of the 
15 multiplex may belong to many programs. Each PES is 

assigned a "packet identifier" (PID) . The PES packets are 
broken into small, fixed-size units called transport 
packets, which may be multiplexed with transport packets 
from other PESs. Coordination and control of the PESs and 
the Programs in the TS is managed via control data called 
Program Specific Information (PSI) . PSI is structured 
according to a section format which differs from and is 
incompatible with the PES packet. PSI is distributed via 
several PIDs, including a base PID (PID 0), used to carry a 
Program Association Table (PAT) listing all Programs in the 
TS, and a PID associated with each Program which carries a 
Program Map Table (PMT) describing the components of the 
Program. PSI syntax also permits the use of PIDs to 
transport private data in formats compatible with PSI, or 
in totally private formats. Other private control 
information may include Entitlement Control Messages (ECMs) 
or Entitlement Management Messages (EMMs) used for 
encryption and authorization, generally referred to as 
conditional Access Information. In addition, the Digital 
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Video Broadcasting (DVB) committee has defined a similar 
structure for transport of Service Information (SI) . 

Private data signals can cover a wide variety of 
purposes, including signals strictly related to broadcast 
5 television applications / such as would be carried in the 
Vertical Blanking Interval (VBI) of an analog television 
signal; signals carrying services ancillary to broadcast 
television applications , such as Electronic Program Guides 
(EPG) ; and signals such as are used for transmitting voice 
10 and/or data in conventional telecommunications networks, in 
both asynchronous and synchronous modes. 

It is apparent that the MPEG2 structures suffer from 
certain difficulties. One class of difficulty emerges when 
one considers the possibility of building modular encoders 
15 and decoders that could be applicable to both TSs and PSs. 
In practice, however, there are significant 
restrictions on such an approach. In the first instance, 
the treatment of control data is very different between the 
PS and the TS. While the control data used to describe a 
20 Program in a PS is in a PES format, the control data used 
in a TS is in an incompatible section format. Thus, the 
interfaces between modules must necessarily be different 
for PS and TS encoders. This difficulty also prevents 
storage of a Program in a form suitable for direct 
25 inclusion in a TS on a device adapted to the PS format. 

Another significant concern is that the PS structure 
imposes restrictions on the types and numbers of program 
elements, or components, which may be part of a Program. 
Such restrictions do not exist in the TS structure. As an 
3 0 example, a Program may contain multiple streams of 
subtitling information that are encoded according to 
International Radio Consultative Committee (CCIR) System B 
Teletext specifications. Each stream of subtitling 
information may be encoded as a PES according to the 
35 procedures defined by the Digital Video Broadcasting (DVB) 
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committee in DVB TM1398 "DVB Subtitling System Working 
Draft 1.0- . Within the TS structure, each such PES of 
subtitling information may be assigned a separate PID and 
may be associated with the Program. However, the PES 
structure is a Private Data PES, and the PS specifications 
only allow one such PES to be associated with a Program. 

A second class of difficulty emerges when one 
considers the variations in the number of components that 
ffi ay be associated with a Program, and the va ^ at " 
their data rates. The Main Profile Main Level of the MP EG 2 
Systems standard permits video elementary streams to be 
transmitted at rates of between 1 and 15 Mbps. A TS may 
include multiple such streams. A decoding device which 
selects such streams from the TS by PID will therefore 
typically be implemented in high speed field-programmable 
gate arrays (FPGAs) or application- specific integrated 
circuits (ASICs) in which the PID is held in a register for 
comparison purposes. The number of PlDs which may be 
processed simultaneously therefore becomes a controlling 
cost element in the design of such FPGAs or ASICs. 
However, since each PID corresponds to a single PES, the 
same register is used regardless of whether the component 
selected is a 15 Mbps video stream or a 1200 bps 
asynchronous data component. The MP EG 2 Systems standard 
25 scheme of making each PID carry a single component can 
therefore induce significant inefficiencies. 

A third class of difficulty emerges when one considers 
certain classes of application for which addressability is 
desirable. For example, it may be desired to address 
30 certain streams, including video and audio streams to 
decoder or a group of decoders. This need may exist 
without regard to the capabilities that may exist for such 
services in the underlying transmission media, for example 
because no such facilities exist in the medium For a 
35 further example, it may be desired to transmit data signals 
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such as datagrams conforming to the Internet Protocol (IP) 
in a private data channel over an MP EG 2 Transport Stream. 
However, a decoder capable of accepting such datagrams must 
first be provided with an IP address. In other broadcast 
5 systems faced with this problem, such as Ethernet, the 

decoder is provided with a native address by a manufacturer 
which may be bound to an Internet address by the systems 
administrator using protocols such as the Reverse Address 
Resolution Protocol (RARP) . Such an approach is not 
10 possible in MPEG2 Systems because no such native address 
scheme is defined for MPEG2 decoders. 

Summary of the Invention 

One object of the present invention is to separate 

15 processing of elementary streams from the multiplexing and 
demultiplexing of such streams into a Transport Stream or 
Program Stream by a common interface based on a universal 
message structure that is compatible with the PES packet 
structure but does not share the limitations of the latter. 

20 Another object is to multiplex multiple elementary 

streams into a format which permits the creation of 
Programs that are compatible with a Program Stream, and 
which permits the use of a single PID to identify multiple 
elementary streams in circumstances when such multiplexing 

25 creates significant efficiencies in use of PIDs in 
Transport Stream decoders. 

A further object of the present invention is to 
address individual signals to individual decoders or groups 
of decoders, without regard to any addressing facilities 

30 that may exist in the underlying transmission medium. 

In accordance with an aspect of the invention, a 
method of providing transport services to a plurality of 
first elementary streams comprises segmenting the 
elementary streams into messages to form a plurality of 

35 first message streams in which each message stream is 
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associated with a particular elementary stream. The first 
message streams are converted into first packetized 
elementary streams. The first packetized elementary 
streams are multiplexed into an output stream. The output 
5 stream can be a Program Stream, a Transport Stream, or one 
of other types of streams including Asynchronous Transfer 
Mode. 

According to another aspect of the invention, a 
plurality of second elementary streams are segmented into 
10 messages to form a plurality of second message streams. 
Data channel identifiers are associated with the second 
message streams, such that. each data channel identifier is 
associated with a particular message stream. The second 
message streams are multiplexed into a second packetized 
15 elementary stream which is then multiplexed with the first 
packetized elementary streams into the output stream. 

According to another aspect of the invention, for a 
Transport Stream, multiplexing the first and second 
packetized elementary streams is achieved by converting 
20 packetized elementary stream packets of certain packetized 
elementary streams into table sections and segmenting the 
sections and the remaining packetized elementary stream 
packets into Transport Stream packets. The Transport 
Stream packets are multiplexed to form the Transport 

25 Stream. , 

According to yet another aspect of the invention, the 
elementary streams can include elementary streams which 
have different data rates. 



30 



R-rief Des ^ipi-ion of the Drawings 

The foregoing and other objects, features and 
advantages of the invention will be apparent from the 
following more particular description of preferred 
embodiments of the invention, as illustrated in the 
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accompanying drawings in which like reference characters 
refer to the same parts throughout the different views. 

Figure 1 is a schematic block diagram of an 
encoder /decoder system in accordance with the present 
5 invention. 

Figure 2 is a flow diagram illustrating a transport 
layer in accordance with the present invention. 

Figure 3 illustrates a Message stream conforming to 
the PES format defined by MPEG 2 . 
10 Figures 4A and 4B illustrates the general content and 

arrangement of a message structure in accordance with the 
present invention. 

Figure 5 illustrates Message Streams having a channel 
number in accordance with the present invention. 
15 Figure 6 illustrates a multiplex of Message Streams 

forming a Private PES in accordance with the invention. 

Figure 7 is a flow diagram illustrating a Transport 
Stream convergence sublayer in accordance with the present 
invention for use in the transport layer of Figure 2. 
20 Figure 8 illustrates a Message Stream which includes 

section formatted information. 

Figure 9 illustrates the format of private data 
descriptors for use in the message structure of Figures 4 A 
and 4B in accordance with the present invention. 

25 Description of th e Preferred Embodiment 

The present invention provides for the adoption of the 
principles of protocol layering specified by the Open 
Systems Interconnection (OSI) Reference Model defined in 
standard ISO/IEC 7498 for use in systems providing MP EG 2 

3 0 PSs and TSs. The MPEG 2 Systems protocol architecture is 
positioned at the Transport Layer of the OSI Reference 
Model. In the present invention, three sublayers are 
provided: 
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s^mngntatioP sublayer. In this sublayer, an 
elementary stream Protocol Data Unit (PDU) is accepted 
across a Service Access Point (SAP) to the sublayer as 
a transport service Service Data Unit (SDU) and is 
5 segmented into units called Messages. A Message is a 

generalization of a PES Packet. The Messages can be 
optionally addressed. The Message is the segmentation 
sublayer PDU. For each elementary stream input to the 
segmentation sublayer, a single PES is created within 
0 the sublayer. The segmentation sublayer also permits 

the identification of Transport Service Connections 
between the input SAP .and the output SAP, called Data 
Channels, by means of a channel identifier. 
M,oHnl e yina sublayer . In this sublayer, a set of 
L5 input Data Channels may optionally be multiplexed into 

a single PES. Messages are both the PDUs and SDUs of 
the multiplexing sublayer. 

g SIS£s£ asnss sufelayex . In this sublayer, the various 
PESs are multiplexed using a format specific to the 
20 transport system being used. 

In the preferred embodiment, two variants of the 
convergence sublayer are defined to support the PS and TS 
defined by MP EG 2 . Additional convergence sublayers could 
be defined for other contexts, e.g., for transport of PESs 
25 using Asynchronous Transfer Mode (ATM) . In the variant of 
the Convergence sublayer used for a PS, referred to as the 
PS convergence sublayer, PES packets are multiplexed 
directly into a PS, as defined by the MPEG 2 Systems 
standard. The variant of the Convergence sublayer used for 
3 0 a TS, referred to as the TS Convergence sublayer, is in 
turn comprised of three sublayers: a format convergence 
sublayer for converting certain PESs to the Table Section 
formats required by MPEG2 Systems for certain purposes, 
notably transport of PSI; an additional segmentation 
sublayer for segmenting PES packets into small, fixed size 
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Transport Packets; and a multiplexing sublayer for 
multiplexing Transport Packets. 

The present invention is based on the following 
principles: 

5 l. Each elementary stream input to a transport service is 
output as an identical single elementary stream. The 
association between the input and output elementary 
streams defines a Data Channel. Each Data Channel 
uniquely identifies an association between the TSAP 
10 (Transport SAP) at the source and the TSAP at each 

decoder. Elementary streams may be freely associated 
with Data Channels. 

2. The Transport Layer is decomposed into sublayers. The 
interfaces between the sublayers is based on the PES 

15 syntax defined by the MPEG2 Systems standard, since 

PES is the common element linking MP EG 2 elementary 
streams with both the Transport Stream and Program 
Stream variants. An extension of the PES packet is 
used to transport all data types in the system in a 

20 Universal Messaging Structure referred to earlier as a 

Message. The Message also provides the structures 
required for addressability. 

3 . The syntax applying to the contents of a Message 
depends on the type of elementary stream. Each 

2 5 Message therefore is assigned a Message Type 

identifying the message syntax. The message type acts 
as a selector for the Presentation Layer Entity needed 
to process the message (e.g. video , audio , VBI) . 
To the extent that a message type is present in a 
30 Message, the Message is self-descriptive regarding the 

presentation layer processing that is required. Since it 
is possible in the future that PESs may be processed by 
transport services having a non-MPEG2 structure, such as 
ATM, and may even be processed in isolation from other 

3 5 elementary streams, Messages should be self-descriptive 
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with respect to presentation layer parameters, where 
possible. 

Referring now to Figure 1, a packet-based 
communication system is shown which embodies the principles 
of the present invention. The system 10 includes an 
encoder 12 and a decoder 18 connected by a transmission 
path 28. The transmission path 28 can be any transmission 
facility, such as a fiber optic link or a 
transmission uplink/ downlink. Elementary streams 24 
control information 23 input to the encoder 12 are 
converted to a single output stream 34 and transmitted over 
path 28 to the decoder 18. The decoder 18 reconver ts the 
output stream to the constituent elementary streams 32 and 
control information 31. The encoder 12 includes a 
Packetized Elementary Stream (PES) 
xnultiplexer 16. The elementary streams 24 and 
information 23 input to the PES packetizer are converted 
PES streams 26. The multiplexer 16 multiplexes the PES 
streams to form the output stream 34. 

The decoder 18 includes a demultiplexer 20 and a PES 
packetizer 22. The demultiplexer 20 
output stream 34 to extract PES streams 30. The 
packetizer 22 converts the PES streams to the -stxtuent 
Elementary streams 32 and control information 31 The 
5 packetizer 14 and the PES de-packetizer 22 operate in 

accordance with the MPEG 2 Systems standard in : preferred 
embodiments, the mux/demux 16, 20 operate with Tra nspor t 
Streams or Program Streams as the output s^ . J» a 
alternate embodiment, the multiplexer and ^ 
10 20 can operate to provide non-MPEG streams, such as 

formatted stream for output stream 34. T<so/1EC 
With reference to the concepts defined in ISO/IK 
standard 7498, an MPEG2 encoder and decoder 
unidirectional broadcast transport services between the 
35 encoder and decoders. The transport services ensure 
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transparent transport of elementary streams between the 
encoder and decoder. MPEG2 defines this process for video 
and audio elementary streams. Figure 2 illustrates the 
generalized transport process embodied in the present 
5 invention. 

In Figure 2, the transport layer of ISO/IEC 7498 is 
used to provide unidirectional services only. The 
transport layer is decomposed into three sublayers in the 
encoder and decoder. Figure 2 illustrates the first two of 
10 these sublayers only which correspond to functions 

performed by the PES packetizer 14 and PES de-packetizer 2 2 
(Figure 1) . The encoding process is illustrated on the 
left side and the decoding process on the right side of 
Figure 2 . 

15 In the encoding process, the input to the encoder is 

provided as one or more elementary streams 102. The 
structure of these elementary streams is defined elsewhere. 
The coding of certain types of elementary streams, notably 
video and audio, are defined by the MPEG2 video and audio 

20 standards specified in ITU-T Rec. H. 262 j ISO/IEC 13818-2 and 
ISO/IEC 13818-3. Each elementary stream is provided to the 
encoder across the transport service interface 104. The 
TSAP 106, 108 identifies the endpoint of a Data Channel. 
The first sublayer of the transport layer is the 

25 segmentation sublayer. In the segmentation sublayer entity 
in the encoder , the contents of the elementary streams are 
processed and divided into Messages. The structure of the 
Message in the present invention is considered in further 
detail below. Each elementary stream is thereby converted 

30 into a Message Stream. Certain Message Streams 103, 
representing the content of Data Channels carrying 
elementary streams defined by MPEG 2 are identical to MP EG 2 
PESs; the identity of the associated Data Channel is not 
explicitly preserved. Figure 3 illustrates a Message 

35 Stream 103 as defined by MPEG 2 . An MPEG2-def ined 



PCT/US97/01093 



WO 97/28652 



-12- 



elementary stream ,0,. such as . expressed audio or video 
atrem i3 there shown segmented into Messages 158. Each 

;« ^ » » Pa =- h e.der 

X« as defined for the PES packet structure in the MPEG2 

'""Tef erring again to Figure a. in certain other Message 
Streams X09. representing the content of Data Channels 
carrying elementary streams not defined by MPEG2 , the 
identity of the Data channel may optionally be preserved by 
, an'idenLfier called e channel ^^'^ " ' 
field in the Message format. In Figure 2, 

identity of the Data Channel 
Channels are shown with the identity o Messa _ es 
preserved. The Message Stream 109 comprises ******** 
having the structure illustrated in Figures 4A and 4B. A 
■ Message 1,3 includes a PES pacXet header --P^ 
182. The Message Header 180 conforms to an MPEG2 defm 
Private Stream 1 PES PacKet format which includes a 

~ 1W The structure of Message 

pes_ P rivate_data field 184. The shQuld 

Header 180 is described in further detail below. 
J0 be noted that a Message can also be based on a 

Private.Stream 2 PES PacKet format in which case private 
data forms the first part of the payload. 

It is important at this point in the discussion to 
note that a channel number can be stored in the 
25 pes private data field 184. Figure S illustrate. . Message 
Streams 109a, 109b having the Message structure 178^ 
Message includes a channel number 188 unique to each 
Message Stream. Certain Messages Streams, and certain 
Messages within individual Message Streams, may be 

30 addressed at this *<>^ ^ encoder , Messa ges 

Referring again to Figure 2, 
are passed by the segmentation sublayer 
sublayer, the Multiplexing sublayer, across th 
interface 110. In this sublayer, all Message "reams are 
35 transformed into PESs by the Multiplexing sublayer entrty. 
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If the Messages in a given Message Stream do not retain the 
channel number, identifying their associated Data Channel, 
they cannot be multiplexed and are passed to the 
Convergence sublayer, as shown in 112. Optionally, as 
5 shown in 114, Messages from different Data Channels may be 
multiplexed into a single PES, conforming to the syntax of 
a Private PES 116 as defined by MPEG 2 , which is then passed 
to the Convergence sublayer. Figure 6 illustrates the 
structure of the Private PES 116. The Private PES 116 is a 

10 stream of messages 178 multiplexed from Message Streams 
109a and 109b. The messages 178 can be distinguished by 
the channel number 188. 

In Figure 2, Message Streams are shown being 
multiplexed into a unique MP EG 2 private PES. Nothing in 

15 the present invention precludes the creation of multiple 
MPEG2 Private PES, unless constrained by the capabilities 
of the Convergence sublayer. The structure of Convergence 
sublayers are discussed in further detail below. 

It will be understood that the interface between the 

20 Multiplexing sublayer and the Convergence sublayer at all 

times consists of an MPEG 2 PES. This is not true for MPEG2 
itself, which in particular defines a second class of 
interface, the so-called MPEG2 sections, for use of 
transport of control information and private data in an 

25 MP EG 2 TS. It is a principal object and a feature of the 
present invention that the interface to the Convergence 
sublayer is at all times an MPEG2 PES, so that different 
Convergence sublayers may be substituted for each other, 
with the only impact upon- the Segmentation and Multiplexing 

30 sublayers, if any, being an adjustment of the set of Data 
Channels which are multiplexed into a set of MPEG2 Private 
PESs. 

It will also be understood that the data capacity of 
the MPEG 2 Private PESs created in the Multiplexing sublayer 
35 of the encoder 116 will be approximately equal to the sum 
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* 4-v,« individual Data Channels 108. In 
of the capacities of the individual ua 

sy stems in which the transport service .us, carry a number 
of elementary streams of widely »ri,»t data rates, the 
invention nay he used to aggregate a number of low-rate 
elementary streams into a single high-rate PES , thereby 
: :lmi3in Y g the resources reguired to identify ^ ™ 

• . CTr nr VLSI implementations 01 
low-rate data streams in ASIC or vu,i i y 

components of the decoder. 

Each PES passed across the interface between the 

Multiplexing sublayer and the Convergence sublayer in the 
encoder 112 is recovered in the Convergence sublayer entity 
in the decoder and passed across the corresponding 
interface 128 to the Multiplexing sublayer entity in the 
decoder. If the PES transports an MPEG-defined el-entarv 

^ Jirerrlv to the segmentation 
stream 119 then it is passed directly to * 

sublayer in the decoder. If an MPEG 2 Private 
transports a multiplex of elementary streams ™* 
according to the present invention, as described above, 
then the multiplex is passed to a demultiplexer within the 

, multiplexing sublayer entity of the decoder 1 20^ Withi 
the demultiplexer, the Data Channel 122 *** oc ™ 
each Message is identified, and the MPEG2 Private PES is 
decomposed into the constituent Message ^ 
Message Streams are not addressed, or, to the extent 

5 addressed Messages within these ^"""^ . 

addressed to the *^.^™>£^ Z^~ 
member of a group of decoders, they are then 
into their corresponding elementary streams 126, which my 
he defined by MP EG 2 or may be other elementary J -reams ^and 
I. are passed across the transport service interface in 

decoder 124. . H 

Many convergence sublayers may be def med In the 
present description of the preferred embedment of the 
invention, only two ere considered: the MP^ PS 
3 5 convergence Sublayer and the «PES2 TS Convergence sublay 
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It will be understood that other Convergence Sublayers can 
be defined, e.g. for the transport of PESs over 
Asynchronous Transfer Mode (ATM) . 

The MP EG 2 PS Convergence Sublayer causes a multiplex 
5 of PESs to be created which is a Program Stream conforming 
to the syntax defined by MPEG. This sublayer is completely 
defined by the MPEG2 specifications relating to the 
creation of Program Streams within the MP EG 2 Systems 
standard, and need not be further discussed here. It will 

10 be understood that the services of the multiplexing 
sublayer defined above permit many data streams to be 
combined into a single MPEG2 Private PES. Since the MPEG2 
PS only permits the presence of two Private PESs, each of a 
different defined PES structure, it is an advantage of the 

15 present invention that a PS created using it can carry many 
more data streams than a PS defined by MPEG2 . For example, 
this facility can be used to transport teletext, closed 
captioning, test and reference signals intended for output 
in the VBI as well as any number of synchronous or 

20 asynchronous data streams in a PS, while the structure 
defined by MPEG2 permits the transport of at most one of 
these in a Private PES of type Private_stream_l or 
Pr i vate_s treara_2 ♦ 

The MPEG 2 TS Convergence Sublayer causes a multiplex 

25 of PESs to be created which is a Transport Stream 

conforming to the syntax defined by MPEG2 . It is apparent 
that the interface defined by the present invention between 
the Multiplexing sublayer and any Convergence sublayer is 
the PES structure defined by MPEG2 . The rules defining the 

3 0 creation of Transport Streams within the MPEG 2 Systems 

standard completely cover the procedures required to create 
a TS from a set of input PESs, and need not be further 
discussed here. However, it is also necessary for the 
MPEG2 TS Convergence Sublayer to convert certain input PESs 

35 into the format expected within a TS, specifically to 
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extract MPEG2 PSI, DVB SI, and private sections such as 
ECMs and EMMs which are defined in formats specific to the 
TS. 

The structure of the Transport Stream Convergence 
5 Sublayer is shown in Figure 7. The processes performed in 
an encoder are shown on the left hand side of the figure. 
The processes performed in the decoder are shown on the 
right hand side of the figure. These processes generally 
correspond to the functions performed by the mux/demux 16 
10 and 20 respectively (Figure 1) . 

The TS convergence Sublayer is itself constructed of 
three sublayers: the TS Format Convergence Sublayer, the TS 
Segmentation Sublayer, and the TS Multiplexing Sublayer, 
in the encoder, the TS Format Convergence Sublayer is 
15 responsible for transforming PES packets received from the 
Multiplexing sublayer into the formats (PES packets or 
sections) expected in the MP EG 2 Transport Stream. In 
certain instances, the TS Format Convergence Sublayer may 
also demultiplex an input PES into multiple streams whose 
20 separate identity is required by the MPEG2 TS structure, by 
DVB, or by third parties. The TS Segmentation Sublayer is 
responsible for segmenting the input streams into fixed 
size Transport Stream packets of 188 bytes. The TS 
Multiplexing Sublayer is responsible for combining the TS 
25 packets in the several input streams into a single TS. 

The functions of the TS Segmentation Sublayer and the 
TS Multiplexing Sublayer are fully defined by those 
sections of the MPEG2 System specifications defining the 
structure of a TS, and will only be briefly considered 
30 here. The TS Format Convergence Sublayer is a feature of 
the present invention. 

With reference to Figure 7, the functions of the TS 
convergence Sublayer in the encoder will now be described. 
All PESs created by the Multiplexing Sublayer are passed to 
3 5 the TS Format Convergence Sublayer. For purposes of 
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managing and control the TS, at least one PES 132 consists 
of Messages comprising the MP EG 2 PSI for the TS, and, if 
appropriate, at least one PES 132 consists of Messages 
comprising the DVB SI for the TS. Other PESs 130 includes 
5 any PES other than the PESs 13 2 which convey MP EG 2 PSI or 
DVB SI intended for use in the current Transport Stream* 
The PESs 130 include MPEG-defined PESs as well as other 
private PESs* 

The PES 132 are illustrated in Figure 8. The PES 132 

10 there shown includes a PES packet header 160 and payload 
162 as defined for the PES packet structure in the MPEG 2 
standard. The PES 132 is formed in the segmentation 
sublayer from an elementary stream which comprises a number 
of section PDUs 210. Each section PDU 210 is encapsulated 

15 into a Message payload 162. The same process can be used 
to convert Internet Protocol datagrams into Messages. 

A PES in which the PES packet structure may be 
preserved intact within the TS 136 is passed without 
processing to the TS Segmentation Sublayer wherein the PES 

20 packets are segmented into multiple Transport Packets as 
defined by the MP EG 2 Systems standard. A PES which 
contains data which is intended for transport as MP EG 2 
Private Sections 134 is processed within the TS Format 
Convergence Sublayer such that the MP EG 2 Private Sections 

2 5 are extracted from the PES Packets and are passed to the TS 
Segmentation Sublayer wherein the Private Sections are 
segmented into multiple Transport Packets as defined by the 
MPEG 2 Systems standard. In each case, the sequence of 
Transport Packets created from the PES or the stream of 

30 private sections are assigned a PID 14 0 for purposes of 

multiplexing within the TS Multiplexing Sublayer using the 
procedures specified by the MP EG 2 Systems standard. 

Since the MPEG 2 Systems standard and DVB require that 
each type of control table be assigned to a separate PID, 

35 the PES(s) input to the TS Format Convergence Sublayer 
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which contain the information used to create these tables 
132 are demultiplexed within the TS Format Convergence 
Sublayer by a demultiplexer 138 such that each output 
stream from the demultiplexer 138 contains all and only the 
5 control table sections required by MP EG 2 Systems and/ or 
DVB, as appropriate, in the format required by MPEG2 
Systems and/or DVB. Each output stream is then passed to 
the TS Segmentation Layer, wherein the table sections are 
encapsulated in a sequence of Transport Packets and 
10 assigned the PIDs reserved for their use by MPEG 2 Systems 
and/ or DVB, and thence to the Multiplexing Sublayer. 

The resultant TS is then passed to the network layer 
of the system across the network service interface 14 2 
whence it is passed to the decoders and recovered across 
15 the network service interface in the decoders 144, exactly 
as in MPEG 2 Systems. 

Any TS created by an encoder embodying the present 
invention is fully conformant with MPEG 2 Systems 
specifications. Accordingly, it can be processed both by 
20 decoders that embody the present invention and by decoders 
that do not embody the present invention. The processing 
performed by decoders embodying the present invention is 
illustrated with reference to the right hand side of 
Figure 7. 

25 The decoder recovers the TS across the network layer 

interface 144 and demultiplexes the constituent streams 
within the TS multiplexing sublayer by PID 145, in 
conformance with MP EG 2 Systems. Within the TS Segmentation 
Sublayer, the PES Packets, Private Sections, and MPEG 2 or 

30 DVB Table Sections are recovered from the Transport 

Packets, in conformance with the specifications of MP EG 2 
Systems or DVB as appropriate. The resultant streams are 
then passed to the TS Format Convergence Sublayer. 

If the recovered stream is a PES in PES Packet format 

35 146, then no further processing need be applied, and the 
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PES may be passed directly to the Multiplexing Sublayer for 
further processing as described above. 

If the recovered stream is a sequence of MPEG 2 Private 
Sections 148 then the TS Format Convergence Sublayer 
5 reformats the Private Sections into Messages for transfer 
to the Multiplexing Sublayer. The same processing is 
applied to recovered streams consisting of MP EG 2 or DVB 
Table Sections 150. Each Message Stream is assigned a 
channel number to identify the associated Data Channel. In 

10 addition, the TS Format Convergence Sublayer may optionally 
combine several of the various recovered message streams 
into a single MPEG 2 Private Stream 152 for transfer to the 
Multiplexing Sublayer. This process is illustrated in 
Figure 7 with particular respect to MPEG2 and DVB Table 

15 Sections. 

It will be apparent that the processing performed by 
the TS Format Convergence Sublayer permits MPEG 2 PSI Table 
Sections, DVB SI Table Sections and Private Sections to be 
represented as PESs. By this means such information can be 

20 included in a PS, for example for storage or retrieval 
purposes. It will further be apparent that a stream of 
MPEG 2 PSI Table Sections and/or DVB SI Table Sections can 
be passed through an MPEG2 TS as an MPEG2 Private PES, 
allowing the Tables to be recovered at some remote point, 

25 for example in a remultiplexer which extracts several 
Programs from several input TSs to create a new TS. By 
this means the MPEG2 PSI Tables and DVB SI Tables required 
by the remultiplexer can be provided from a remote source 
without affecting the operation of any of the input TSs. 

30 Other applications can be provided using these 

capabilities, which are not defined or possible using the 
structures defined by MP EG 2 Systems. 

The system architecture of the present invention, 
described in the preceding paragraphs, requires a Message 

35 format that has certain particular characteristics: 
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10 



A Message is completely compatible with a PES Packet 
and permits all facilities permitted by the PES 

Paclcet • 

Messages are capable of transporting elementary 
streams defined by MP EG 2 and elementary streams which 
are not defined by MPEG2. 

Messages identify the type of elementary stream 
carried by the Messages, so that each Message may be 
delivered to the appropriate Presentation entity in 
the decoder for processing by means appropriate to the 
type of elementary stream. 

Messages can, if appropriate, identify the Data 
Channel associated with the Messages, so that multiple 
Data Channels may be combined into a single MPEG2 
15 Private PES within the Multiplexing Sublayer. 

Messages can, if appropriate, include an address 
identifying the decoder or group of decoders intended 
to receive the Message. 

For Messages carrying data intended to be transported 
within a TS as Table Sections (e.g. for MP EG 2 or DVB 
Tables or for Private Sections) , procedures must be 
defined to recover the various Sections from the 
Messages within the TS Format Convergence Sublayer : 
The structure of a Message in the preferred embodiment 
25 may therefore be defined by reference to an MPEG2 PES 

Packet, as illustrated in Figures 4A and 4B as discussed 

above. , . _ 

A Message is an MP EG 2 PES Packet. When used in the 
preferred embodiment to implement the features of the 
30 present invention, a Message is specifically formatted as 
an MP EG 2 PES Packet of type Private_stream_l As noted 
earlier, the Message comprises a Message Header 180 that is 
the PES packet header of the Message, and a Message Pay load 
182 that is the PES packet payload of the Message, each 
35 formatted as discussed further below. 
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The Message Header 180 is defined as having certain 
specific restrictions with relationship to the general 
structure of PES Packets defined by the MPEG 2 Systems 
standard, as follows: 
5 The stream_id field has the value OxBD (MPEG2 

Private_Stream_l) . 

The maximum value of the PES_packet_length field is 
dependent on the type of elementary stream data carried in 
the Message and is determined on a case by case basis. 

10 The PES__extension_f lag is set to indicating that 

PES extension fields defined by MPEG2 are present. 

The PES_private_data_f lag set to indicating that 

a PES_private_data ( ) extension field 184 is present. The 
encoding of the PES_private_data ( ) field is specified 

15 further below. 

The Message Header 180 is terminated by at least one 
stuffing octet, to ensure that a false occurrence of the 
MPEG 2 PES start_code is not accidentally created straddling 
the boundary of the Message Header 180 and the Message 

20 Payload 182. All other fields of the Message Header 180 

may be freely encoded as defined by MPEG2 for a PES Packet, 
and as appropriate for the type of elementary stream data 
carried in the message. 

In order to prevent false occurrences of the MPEG2 PES 

25 start_code 0x000001, the contents of all Message Payloads 
are subjected to a bit-stuffing procedure prior to 
inclusion in the Message. The procedure defined here for 
the preferred embodiment is one of many possible such 
bit-stuf f ing procedures . 

30 The intended payload is scanned for a sequence of 22 

4 0* bits. Whenever such a sequence is found, a '1' bit is 
inserted into the payload immediately following the 2 2nd 
'0' bit. If the resultant payload is not an integral 
number of octets, then the minimum necessary number of 4 1' 

35 bits is appended to the payload to cause the payload to be 
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an integral number of octets. If there have been N bit 
insertions, then the number of *l f bits to be appended is 
given by: 

Number of *1' bits = (8 - N mod 8) mod 8 
5 This procedure has certain consequences for certain 

fields of the Message Header 180 , as follows: 
The PES_packet_length is calculated after completion of 
bit-stuffing- The encoder ensures that the 
PES_packet_length so calculated does not exceed 6553 5 
10 octets. All decoders of this preferred embodiment may be 
capable of accepting Messages with a PES_packet_length of 
at least 4095 octets after bit-stuff ing. The actual 
maximum size of a Message is dependent on the type of 
elementary stream transported in the Message. 
15 The previous_PES_packet_CRC, if present, is calculated 

over the contents of the previous Message after 
bit-stuffing has been performed. 

The reverse procedure is performed when extracting a 
Message Pay load 182 from a received Message. The received 
20 payload is scanned for a sequence of 22 *0' bits followed 
by a 4 l f bit. Whenever such a sequence is found, the 'T 
bit is discarded, and the count of 4 0' bits restarted. If 
the resultant payload is not an integral number of octets, 
then the minimum number of bits necessary to byte-align the 

2 5 payload is discarded. If there have been N bit removals, 

then the number of bits to be removed from the end is given 
by: 

Number of discarded trailing bits = (8 - N mod 8) mod 8 
If a sequence of 23 4 0* bits is found in the Message 

3 0 Payload, then the Message contents prior to the start of 

the sequence are declared invalid and the partial message 
is discarded. 

The structure of the PES_private_data ( ) field 184 has 
the format shown in Figures 4A and 4B in the preferred 
3 5 embodiment of the present invention. The 128 bits reserved 
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for the PES private data () field as defined in the MP EG 2 
Systems standard is considered as 8 Words each of 16 bits. 
The bits in each Word are identified by a number from 0 to 
15 inclusive, where Bit 0 is the Most Significant Bit (MSB) 
5 of the Word, and Bit 15 the Least Significant Bit (LSB) of 
the Word. 

The MSB of each word 186 is denoted the marker_bit. 
It is set to 4 1 ' , to ensure that the PES start code is not 
accidentally emulated in the PES_private_data ( ) field. 

10 Bits 1 to 15 of the first word 188 constitutes the 

channel_number field. This field identifies a Data Channel 
within the MPEG 2 Private PES. 

Values of the channel number field in the range 0x0000 
to 0X000F inclusive are reserved in the preferred 

15 embodiment for use by the TS Format Convergence Sublayer of 
the decoder to prepend Message Headers to payloads which 
must be processed by the decoder but are delivered through 
a Transport Stream in a non-PES packet format. The 
following values are defined at this time: 

2 0 0x0000: MPEG 2 PSI for current Transport Stream (i.e. 

PID 0, PID 1 and PMT PIDs) 

0x0001: DVB SI for current Transport Stream (i.e. 
PIDs 0x0010-0x0013) 
0x0002 -0X000F : Reserved . 
25 Values greater than OxOOOF may be freely assigned to 

identify Data Channels by the encoder. 

Bits 1 to 15 of the second, third, fourth, and fifth 
word of the PES_private data ( ) field constitute a 60-bit 
field called the decoder_identif ier 190, which may be used 
30 to address a single decoder or a group of decoders 

embodying the present invention. Each decoder is provided 
with a unique identifier of this type at manufacture time. 
Identifiers may be assigned to identify user groups, which 
may be associated with individual decoders by means of an 
35 Address Resolution Protocol. This facility allows services 
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which are otherwise inherently broadcast, such as VBI Data 
or User Data, to be restricted to a closed user group. In 
addition, the address 0x000000000000000 is reserved for use 
as the broadcast address. 
5 With reference to the classical Internet protocol 

architecture, the decoder_identif ier 190 acts as a 
subnetwork address for a "network" based on MP EG 2 PESs. It 
is therefore formally distinct from the physical address 
associated with a particular input port, such as the 
10 decoder's Ethernet address, with which a different 

subnetwork protocol will be associated. However, if a port 
is not otherwise provided with a physical address, the 
decoder_identif ier may act as the port's physical address. 
It may be bound to a decoder's Internet address via an 
15 Address Resolution Protocol. 

Bit 1 of the sixth word of the PES_private_data ( ) 
field 192 constitutes the PES_header_removable field. This 
field identifies whether the Message Header 180 of this 
Message may be removed when inserting the Message Payload 
20 182 into an MPEG2 TS, or other structure created by a 

convergence sublayer. (Without loss of generality, only 
the possibility of an MPEG2 TS will be referred to in 
subsequent discussion.) If the value of this field is set 
to 4 0', then the Message Header 180 is not removed when 
25 inserting the Message into a TS. If the value of this 
field is set to 4 1\ then the Message Header 180 may 
optionally be removed when the Message is inserted into a 
TS. The decision to remove the Message Header 180 of a 
Message with a removable header is made on the basis of 
30 criteria that are external to this invention. If the 
Message Header 180 is removed, then the bit-stuffing 
process performed on the Message Payload 182 described 
above is inverted prior to any subsequent processing that 
may be required. The PES_header_removable bit may be set 
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to '1' on Messages of message_type 0x0000 (MPEG2 PSI) or 
0X0001 (DVB SI) . 

Bits 2-15 of the sixth word of the PES_private_data ( ) 
field 194 constitutes the message_type field. This field 
5 identifies the type of elementary stream data carried in 
the Message Payload 182 of the Message, It is used to 
select the service access point (SAP) for appropriate 
processing in the decoder or other downstream module; thus 
it acts in practice as a conventional Application Protocol 
10 Identifier (API) . Note that the selection of a 

tnessage__type does not necessarily carry any implications 
about the interface across. which the data entered the 
encoder. For example, VBI data could be inserted through a 
user data port in the encoder, but may be inserted in the 
15 VBI lines of the analog video output of a decoder. 

The following values are defined for the message_type 
field in the preferred embodiment: 

0x0000 MPEG 2 PSI 

0x0001 DVB SI 

20 0x0002 Device Management 

0x0003 User data 

0x0004 VBI data 

0x0005 Address Resolution Protocol 

0x0006 Internet Protocol 

25 0x0007-0x3 FFF Reserved 

Other roessage_type values can include, for example, 
private data encoded according to the formats of a third 
party manufacturer. 

Messages of the same type (e.g. DVB SI, or VBI data 
30 for different lines) may be sent oyer multiple Data 
Channels. However, any Data Channel is used to carry 
Messages of precisely one Message type. 

Note that there is nothing in the MP EG 2 Systems 
standard that prevents the structuring of video and audio 
35 PES packets as Messages containing channel identification 
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and message typing data as defined by the preferred 
embodiment of the present invention. However, such an 
implementation is not recommended* MPEG 2 Systems maintains 
the integrity of the elementary stream as a separate entity 
5 down to the message interface, so there is no great 

advantage to augmenting the MP EG 2 PES syntax for video and 
audio packets. Moreover, MPEG 2 Systems provides no means 
to distinguish users of the pes_private_data ( ) field of an 
MPEG2-def ined elementary stream. Different decoders may 

10 interpret this field differently, and certain decoders may 
not accept it at all. For these reasons, channel numbers 
and message types for video and audio should be retained at 
most as elementary stream attributes internal to the 
encoder or decoder. 

15 Bits 1-15 of each of the remaining Words of the 

PES_private data() field 196 are reserved for uses to be 
defined. 

The permitted formats of the Message Payload 182 vary 
according to the value of the message_type field. In order 

20 to support the functionality required for the TS Format 
Convergence Sublayer defined above, the preferred 
embodiment includes definitions of the permitted formats 
for Messages of message type 0x0000 (MPEG 2 PSI) or 0x0001 
(DVB SI) . Formats are defined prior to bit-stuffing of the 

25 Message Payload 182. 

A Message of message_type 0x0000 contains a single 
MPEG2 PSI section (PAT, PMT, or CAT) , as defined by MPEG2 
Systems. Network Information Table (NIT) sections are 
located through the PAT, but are carried in messages of 

30 type 0x0001, if the NIT has been defined by DVB, or of a 
type appropriate to the private syntax, if the NIT has not 
been defined by DVB. 

A Message of message_type 0x0001 contains a single DVB 
SI section as defined by DVB. 
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Th e format of Messages of message_type other than 
0x0000 or 0x0001 may be determined on a case by case basis, 
as each value of message_type is assigned. 

In order to permit access to various Data Channels in 
5 both a TS and a PS, it is necessary to extend the 

structures defined by MPEG 2 Systems to account for the 
various elementary stream types supported by the present 
invention. This is primarily done by defining private 
descriptors defining the contents of a particular Data 
10 Channel. These descriptors can be included in the PMT 

sections in the TS and in the Program Stream Map (PSM) in 
the PS. 

In the preferred embodiment, message streams 
conformant to the present invention are identified within 

15 PMTs or PSMs by stream_id 0x06 (PES packets containing 
private data) . 

Message streams conformant to the present invention 
may share a TS with one or more MP EG 2 Private PESs using a 
different syntax from the syntax defined in this preferred 

20 embodiment. Both streams will be described through 

descriptors in MP EG 2 PSI (and possibly DVB SI) . For all 
parties, the descriptors used will possess a syntax 
regarded as privately defined within MP EG 2 Systems. In 
order to distinguish the Message streams conformant to the 

25 present invention, the associated private descriptors must 
be distinguished from descriptors appropriate to the other 
private descriptors by use of an MPEG 2 

registration_descriptor () identifying the syntax as being 
conformant with the present invention. The appropriate 
30 value of a f ormat_identif ier attribute of the 

registration_descr iptor ( ) can be obtained from the 
appropriate Registration Authority, as required by MP EG 2 
Systems . 

Since neither MPEG2 Systems nor DVB has defined 
35 scoping rules for the registration_descriptor ( ) , the 
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following rule is assumed in the preferred embodiment of 
the present invention. The registration_descr iptor ( ) 
applies to all subsequent MP EG 2 private descriptors 
encountered within the same loop as the 
5 registration_descr iptor () until the next 

registration_descriptor () encountered within the same loop. 
MPEG-private descriptors not preceded by a 

registration_descriptor() identifying the syntax as being 
conformant with the present invention are assumed to have a 
10 syntax defined by DVB* 

At such time as private descriptors are required 
within DVB SI for purposes related to the present 
invention, a value for a private_data_specif ier can be 
obtained from the appropriate Registration Authority. In 
15 each descriptor loop within a DVB SI section containing at 
least one private descriptor related to the present 
invention, both a registrationdescriptor ( ) , defined by 
MPEG2 Systems, and a private_data_specif ier_descriptor ( ) , 
defined by DVB, are supplied preceding the first 
20 occurrence of a private descriptor in the loop. 

In addition to the registration procedures defined 
above, each type of private descriptor is identified by a 
value of the descriptor_tag field. In the preferred 
embodiment of the present invention, descriptor_tag values 
25 are assigned in the range OxCO-OxFE. An example of the 
assignment of values of the descriptor_tag field is as 
follows: 

Descriptor descriptor_tag value 

psi_descr iptor ( ) OxCO 
3 0 dvb_descriptor() OxCl 

tup_descr iptor { ) 0xC2 

user_data_descriptor ( ) 0xC3 

vbi_descriptor ( ) 0*C4 

harp_descr iptor ( ) 0xC5 
35 ipdescr iptor () 0xC6 
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reserved 0xC7-0xFE 

For the purposes of implementing the TS Format 
Convergence Sublayer of the present invention, definitions 
for the psi descriptor () and the dvbjiescriptor ( ) are 
5 required in the preferred embodiment. The psidescriptor ( ) 
signals the presence of MP EG 2 PSI data in a named Data 
Channel. The dvb_descriptor ( ) signals the presence of DVB 
SI data in a named Data Channel. The format of the 
psi_descriptor () and the dvb_descriptor ( ) is shown in 
10 Figure 9. 

Bits 0-7 of the first Word 198 constitute the 
descriptor_tag field as defined by MP EG 2 Systems. In the 
case of the psi_descriptor ( ) , this field is set to OxCO in 
the preferred embodiment. In the case of the 
15 dvb_descriptor () , this field is set to OxCl in the 

preferred embodiment. Bits 8-15 of the first Word 200 
constitute the descriptor_length field as defined by MP EG 2 
Systems. This field is set to 2 in the preferred 
embodiment for the psi_descriptor ( ) and dvb_descriptor ( ) . 
20 For other descriptor types, the value of the descriptor 
length field is at least two. 

Bit 0 of the second Word 202 is reserved and is set to 
4 1* in the preferred embodiment. Bits 1-15 of the second 
Word 2 04 constitute the channel_n umber field. In the case 
25 of the psidescriptor () f this field identifies a Data 
Channel carrying MPEG 2 PSI data in the preferred 
embodiment. In the case of the dvb_descriptor ( ) , this 
field identifies a Data Channel carrying DVB SI data in the 
preferred embodiment. 
3 0 Descriptor formats for other elementary stream types 

are defined on a case by case basis. 

The present invention permits the construction of 
encoders capable of accepting a variety of information 
inputs and creating enhanced MPEG2 Program Streams or 
35 Transport Streams, the components of which may optionally 
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be addressed to individual decoders or groups of decoders; 
transcoders and reroultiplexers capable of accepting MPEG2 
Program Streams and/or Transport Streams, along with other 
sources of input, and generating enhanced MP EG 2 Program 
5 Streams and/ or Transport Streams as output; and decoders 
capable of accepting enhanced MPEG 2 Program Streams and/or 
Transport Streams as input and providing video, audio and a 
variety of other types of information as output. 

The present invention has been described in the 
10 context of a packet-based multiplexed communication system 
that is compatible with the MPEG 2 Systems standard. It 
will be understood, however, that the methods of the 
present invention are not limited to systems that operate 
in accordance with the MP EG 2 Systems standard. Rather, the 
15 present invention may be embodied in any packet-based 

multiplexed communication system in which it is desirable 
to combine one or more streams of video, audio, and other 
data into single or multiple streams suitable for storage 

or transmission. 

20 While this invention has been particularly shown and 

described with references to preferred embodiments thereof, 
it will be understood by those skilled in the art that 
various changes in form and details may be made therein 
without departing from the spirit and scope of the 

25 invention as defined by the appended claims. 
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CLAIMS 



What is claimed is: 



1. In a packet-based communication system wherein 
elementary streams are segmented into packetized 

5 elementary stream packets to form packetized 

elementary streams, a method of providing transport 
layer services comprising the steps of: 

providing a plurality of first elementary 
streams ; 

10 segmenting the first elementary streams into 

messages to form a plurality of first message streams 

converting the first message streams into first 
packetized elementary streams, each first packetized 
elementary stream associated with a particular first 
15 elementary stream; and 

multiplexing the first packetized elementary 
streams into an output stream. 

2. The method of Claim 1 further comprising the steps of 

providing a plurality of second elementary 
20 streams; 

segmenting the second elementary streams into 
messages to form a plurality of second message 
streams, each second message stream associated with a 
particular second elementary stream; 
25 associating a data channel identifier with each 

second message stream; 

multiplexing the second message streams into a 
second packetized elementary stream ; and 

multiplexing the second packetized elementary 
30 stream with the first packetized elementary streams. 



3. 



The method of Claim 1 wherein the output stream is a 
Program Stream. 
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4 . 



The method of Claim 1 wherein the output stream is a 
Transport Stream. 

The method of Claim 4 wherein the step of multiplexing 
first and second packetized elementary streams further 
comprises the steps of: 

converting packetized elementary stream packets 
of certain first packetized elementary streams into 
sections; 

segmenting the sections and the packetized 
elementary stream packets of the remaining packetized 
elementary streams into transport stream packets; and 

multiplexing the transport stream packets into 
the transport stream. 

6. The method of claim l wherein the output stream is an 
15 Asynchronous Transfer Mode Stream. 

7. The method of Claim 1 wherein the elementary streams 
include elementary streams which have different data 
rates. 



10 



8. The method of claim 1 wherein each message is an MPEG 
20 Private_Stream_l Packetized elementary stream packet 

comprising a message header and a message pay load. 

9. The method of Claim 8 wherein the message header 
includes a PES_private_data () extension field. 

10. The method of Claim 9 wherein the PES_private_data ( ) 
25 field includes a channel_nurober subfield for 

associating a data channel with the message stream. 



WO 97/28652 



PCT/US97/01093 



-33- 

11. The method of Claim 9 wherein the PES_private_data ( ) 
field includes a decoder_identif ier subfield for 
addressing a decoder. 

12. The method of Claim 9 wherein the PESprivate data ( ) 
5 field includes a decoder_identif ier subfield for 

addressing a group of decoders. 

13. The method of Claim 11 wherein the decoder_identif ier 
subfield comprises an Internet protocol subnetwork 
address. 

10 14. The method of Claim 9 wherein the PES_private_data ( ) 

field includes a message_type subfield for identifying 
elementary stream data type carried in the message 
pay load. 

15. In a packet-based communication system wherein 
15 elementary streams are segmented into packetized 

elementary stream packets forming packetized 
elementary streams and wherein the packetized 
elementary streams are segmented into transport stream 
packets which are then multiplexed to form a transport 
20 stream, a method of providing transport services 

comprising the steps of: 

converting packetized elementary stream packets 
of certain packetized elementary streams into 
sections ; 

25 segmenting the sections into transport stream 

packets; and 

multiplexing the transport stream packets into 
the transport stream. 
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16. The method of Claim 15 wherein the certain packetized 
elementary streams include program specific 
information or service information. 

17. In a packet-based communication system wherein a 
plurality of elementary streams are multiplexed to 
form an output stream for transmission to a remote 
site, a method of providing transport layer services 
to the elementary streams comprising the steps of: 

segmenting the elementary streams into messages 
to form a plurality of message streams, each message 
stream associated with a particular elementary stream; 

associating data channel identifiers with certain 
message streams, each data channel identifier 
associated with a particular message stream; 

multiplexing the certain message streams into a 
first packetized elementary stream; and 

converting the remaining message streams into 
second packetized elementary streams. 

18. The method of Claim 17 further comprising the step of 
20 multiplexing the first and second packetized 

elementary streams into the output stream. 

19. The method of Claim 18 wherein the output stream is a 
Program Stream. 

20. The method of Claim 18 wherein the output stream is a 
25 Transport Stream. 



15 



21. 



The method of Claim 20 wherein the step of 
multiplexing first and second packetized elementary 
streams further comprises the steps of: 
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converting packetized elementary stream packets 
of certain second packetized elementary streams into 
sections; 

segmenting the sections and the packetized 
elementary stream packets of the remaining packetized 
elementary streams into transport stream packets; and 

multiplexing the transport stream packets into 
the transport stream. 

The method of Claim 21 wherein the sections are table 
sections which include program specific information or 
service information . 

The method of Claim 18 wherein the output stream is an 
Asynchronous Transfer Mode stream. 

The method of Claim 21 wherein each message is an MPEG 
Private_Stream_l Packetized elementary stream packet 
comprising a message header and a message pay load. 

The method of Claim 24 wherein the message header 
includes a PES_private_data ( ) extension field. 

The method of Claim 25 wherein the PES_private_data ( ) 
field includes a channel_number subfield for 
associating a data channel with the message stream. 

The method of Claim 25 wherein the PES_private_data ( ) 
field includes a decoder_identif ier subfield for 
addressing a decoder. 

The method of Claim 25 wherein the PES_private_data ( ) 
field includes a decoder_identif ier subfield for 
addressing a group of decoders. 
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The method of Claim 27 wherein the decoder_identif ier 
subfxeld comprises an Internet protocol subnetwork 
address. 

The method of claim 25 wherein the PE S _private_data (, 
field includes a message_type subfield for identifying 
elementary stream data type carried in the message 
pay load . * 

in a packet-based communication system wherein 
elementary streams are multiplexed to form an output 
stream for transmission to a remote site, a method of 
addressing a decoder at the remote site, comprising 
the steps of: 

segmenting the elementary streams into messages 
to form a plurality of message streams, each message 
stream associated with a particular elementary stream, 
wherein each message includes a message header and 
message pay load; and 

identifying the remote decoder with a decoder 
identifier field in the message header. 

The method of claim 31 wherein the decoder identifier 
comprises an Internet protocol subnetwork address. 

The method of Claim 31 wherein the decoder identifier 
is located in private data field of the message 
header . 

In a packet-based communication system wherein a 
plurality of elementary streams are multiplexed to 
form a transport stream for transmission to a remote 
site, a method of providing transport layer services 
to the elementary streams comprising the steps of: 
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segmenting the elementary streams into messages 
to form a plurality of message streams, each message 
stream associated with a particular elementary stream; 

associating data channel identifiers with certain 
5 message streams, each data channel identifier 

associated with a particular message stream; 

multiplexing the certain message streams into a 
private packetized elementary stream; 

converting the remaining message streams into 
10 second packetized elementary streams; and 

multiplexing the private and second packetized 
elementary streams into the transport stream. 

35. The method of Claim 34 wherein the step of 
multiplexing private and second packetized elementary 

15 streams further comprises the steps of: 

converting packetized elementary stream packets 
of certain second packetized elementary streams into 
sections; 

segmenting the sections and the packetized 
20 elementary stream packets of the remaining packetized 

elementary streams into transport stream packets; and 

multiplexing the transport stream packets into 
the transport stream. 

36. The method of Claim 35 wherein the sections are table 
25 sections which include program specific information or 

service information . 

37. The method of Claim 35 wherein the sections are 
private sections. 



30 



38. 



In a packet-based communication system wherein a 
plurality of elementary streams are multiplexed to 
form a transport stream for transmission to a remote 
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site having a decoder, a method of retrieving the 
elementary streams at the decoder comprising the steps 
of : 

receiving the transport stream at the remote 

site; 

demultiplexing the transport stream into a first 
packetized elementary stream and second packetized 
elementary streams; 

demultiplexing the first packetized elementary 
stream into first message streams and converting the 
second packetized elementary streams into second 
message streams, each message comprising a packetized 
elementary stream packet; 

identifying data channel identifiers associated 
with the first message streams, each data channel 
identifier associated with a particular first message 
stream; and 

reassembling packetized elementary stream packets 
of the first and second message streams into 
2 0 elementary streams. 



10 



15 



39 



25 



30 



In a packet-based communication system wherein a 
plurality of elementary streams are multiplexed to 
form a transport stream for transmission to a remote 
site, apparatus for providing transport layer services 
to the elementary streams comprising: 

means for segmenting the elementary streams into 
messages to form a plurality of message streams, each 
message stream associated with a particular elementary 
stream; 

means for associating data channel identifiers 
with certain message streams, each data channel 
identifier associated with a particular message 
stream; 



WO 97/28652 



PCT/US97/01093 



-39- 

means for multiplexing the certain message 
streams into a private packetized elementary stream 

means for converting the remaining message 
streams into second packetized elementary streams; 
5 means for multiplexing the private and second 

packetized elementary streams into the transport 
stream. 
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